Motivation-Learning and Aging 1 Toward a Three-Factor Motivation-Learning Framework in Normal Aging

The common belief that motivation involves simply “trying harder” is at best simplistic and at worst is inaccurate. In this Chapter we highlight the importance of studying motivation at multiple levels to better understand the conditions that support effort-based learning strategies relative to automatic learning strategies (global motivation, local motivation) and demonstrate how effective these approaches are when seeking immediate or long-run rewards (task-directed). Global motivations represent the overall goal of approaching positive outcomes (e.g., a raise or bonus) or avoiding negative outcomes (e.g., a demotion or pay cut). Local motivations represent the immediately relevant goal of approaching positive feedback (e.g., maximizing rewards or making someone happy) or avoiding negative feedback (e.g., minimizing punishments or avoiding making someone angry). Global and local motivational states interact to influence the competition between executive and automatic strategies. An approach-approach or avoid-avoid match shifts the bias toward cognitive control whereas a mismatch shifts the bias toward habitual procedural processing. The effects of each of these strategies during learning depend on taskdemands. Task-directed motivation reflects whether the task is goal-directed, relying heavily on cognitive control processes, or is reward-based, relying on habitual procedural processes. Thus, performance in a task critically depends on a complex three way interaction between local, global, and task-directed motivation. We extend this framework to normal aging and provide evidence from two studies that normal aging is associated with a bias toward reward-based processing. In addition, we argue that computational modeling techniques are underutilized. Introduction Most behavior stems from motivation. As we maneuver through the environment we chose actions from a large repertoire of behaviors. These behaviors are strongly affected by our learning history, but also by our current motivational state to approach positive outcomes or avoid negative outcomes. For example, one could be motivated to be on time for a meeting or to avoid being late for a meeting. Similarly, one could be motivated to achieve a particular score on an exam or avoid falling below a particular score. The goal is the same, but the motivational frame through which one views the goal is different. The approach-avoidance dichotomy is wellestablished in the traditional psychology of motivation (Aarts, Gollwitzer, & Hassin, 2004; 1 This research was funded by NIA grant AG043425 to WTM and DAW, NIDA grant DA032457 to WTM, and a University of Texas Powers Graduate Fellowship to MAG. Correspondence should be addressed to W. Todd Maddox ([email protected]). Motivation-Learning and Aging 2 Ferguson & Bargh, 2004; Fishbach, Friedman, & Kruglanski, 2003; Gray, 1970, 1985; Higgins, 2000; Hull, 1943; Lewin, 1935; Mowrer, 1960; Murty, LaBar, Hamilton, & Adcock, 2011). Perhaps surprisingly, most cognitive research focuses on information processing and its effects on learning and behavior with little attention paid to the factors that drive or motivate one to act. Interestingly, this artificial separation of motivation research from learning research was not present in the 1950’s and 1960’s (Miller, 1957, 1959; Young, 1959). However, as psychology became more divided and area-driven, learning research became the domain of cognitive and animal psychologists, whereas motivation was primarily studied by social and educational psychologists. In many ways, the cognitive neuroscience revolution that began in the 1980’s and 1990’s provided the necessary spark for bringing research on learning and motivation back together. Cognitive neuroscience research makes clear that the brain does not distinguish between “motivational” brain systems and “learning” brain systems. In fact, some of the most important brain regions for learning such as the prefrontal cortex, the anterior cingulate and the caudate nucleus are known to be involved in motivation, affect, and personality (Baldo & Kelley, 2007; Belin, Jonkman, Dickinson, Robbins, & Everitt, 2009; Berridge, 2003, 2007). In addition, detailed neurobiological theories are beginning to take hold that postulate specific interdependencies between “cognitive” and “motivational” brain regions (Ashby, Isen, & Turken, 1999; Bechara, Damasio, & Damasio, 2000; Bechara et al., 2001; Chiew & Braver, 2011; Jimura, Locke, & Braver, 2010; Murty, Labar, & Adcock, 2012; Pickering, 2004; Spielberg et al., 2011; Spielberg et al., 2012). Thus, it is clear that motivation and learning are intimately related and advances in one field should be associated with advances in the other. Organization of the Chapter The overriding aim of this chapter is to explore the motivation-learning interface broadly, but also with applications in healthy aging. First, we begin by asking the fundamental question, “What is motivation and how is it defined?” We conclude that the layman’s definition, and often the implicit scientific definition, is limited in scope. After reviewing common definitions of motivation, we explore more rigorous definitions and conclude that motivation can operate at a global or at a local level with each having an approach and an avoidance state. The interaction between the two states is proposed to directly affect the availability of cognitive resources and subsequent behavior. Global motivation, or the big-picture intent of behavior, can involve approaching positive outcomes such as a promotion or a bonus, or involve avoiding negative outcomes such as avoiding a demotion or pay cut. Local motivation, or the immediate intent of behavior, can involve maximizing performance indices such as the number of trials performed correctly or the number of points earned, or involve avoiding losses such as the number of errors or the number of points lost. Global and local motivators are often present at the same time and understanding how this influences processing biases is critical in predicting learning outcomes. Next, we explore the learning side of the motivation-learning interface and argue that task demands interact with processing strategies as a form of task-directed motivation. Contemporary cognitive psychology acknowledges dissociable learning systems that influence task-directed motivation (Ashby, Alfonso-Reese, Turken, & Waldron, 1998; Ashby, Paul, & Maddox, 2011; Blanco, Otto, Maddox, Beevers, & Love, 2013; Daw, Gershman, Seymour, Dayan, & Dolan, 2011; Glascher, Daw, Dayan, & O'Doherty, 2010; Hayes & Broadbent, 1988; Kendler & Kendler, 1970; Sloman, 1996; Worthy, Otto, & Maddox, 2012). Sometimes the task is such that effortful cognitive control processes and goal-directed behavior optimize performance. At other Motivation-Learning and Aging 3 times, the task is such that automatic, habitual and procedurally-driven behavior optimizes performance. In this section we bridge a dual-learning systems framework with motivation while exploring the underlying neural systems. We provide strong evidence suggesting a complex three-way interaction between the global motivation (approach or avoidance goals), the local motivation (valence of trial-by-trial feedback: gains or losses) and the learning system (goaldirected or reward-directed). Here the interaction of global and local motivation influences which task-directed system is dominant (Figure 1 described in detail below). This has very different implications in modulating goal-directed behavior and reward-directed behavior. Although we describe the three aspects of motivation separately, it is important to emphasize that we espouse highly interactive systems whose effects on behavior are not independent. Finally, we extend these concepts to healthy aging and briefly review two studies from our lab that explore the motivation-learning interface in older adults. Healthy older adults demonstrate differences in baseline levels of task-directed motivation where executive function is diminished limiting their ability to carry out complex goal-directed behavior and exaggerating their dependence on automatic processes (Figure 2). These applications explore age-related changes in the way global motivation influences task-directed motivation during decisionmaking. Critically, we use behavioral tasks that are identical in all respects except the nature of the optimal learning system. We take advantage of a novel computational modeling approach that allows us to quantify the effects of motivation on dual-processes strategies. Finally, we summarize the complex interaction of global, local and task-directed motivation, offer some conclusions, and suggest a number of lines of future research. What is Motivation and How is it Defined? It is commonly thought that motivating someone involves getting them to “try harder”. Although this definition captures some important aspects of motivation, it is too simplistic and is lacking in at least two important ways. First, defining motivation as “trying harder” implies an effortful, controlled task-directed motivation that is frontally mediated. As we will see in the next section, the effects of motivation are more complex with some motivational states enhancing frontal function and others attenuating frontal function. Second, this definition implies that “trying harder” enhances performance, but this is not always the case. In fact, at times decreasing available effortful cognitive control resources through the introduction of a dual-task has no effect on performance or even enhances performance (Filoteo, Lauritzen, & Maddox, 2010; Maddox, Ashby, Ing, & Pickering, 2004; Waldron & Ashby, 2001; Worthy, et al., 2012; Zeithamova & Maddox, 2006, 2007). Thus, trying harder helps in some cases, but hinders in others. Global and Local Aspects of Motivation The motivation literature makes a distinction between global approach and global avoidance goals (Carver & Scheier, 1998; Fishbach, et al., 2003; Lewin, 1935; Maddox & Markman, 2010; Maddox, Markman, & Baldwin, 2006b; Markman & Brendl, 2000; Miller, 1957; Murty, et al., 2011). Goals with positive states that one wishes to achieve are called approach goals (e.g., a raise), whereas goals with negative states that one wishes to avoid are called avoidance goals (e.g., a demotion). Local motivation can involve maximizing performance indices such as the number of trials performed correctly or the number of points earned, but can also involve avoiding losses such as the number of errors or the number of points Motivation-Learning and Aging 4 lost. Global and local motivational states can be manipulated independently and vary broadly in the real-world and in the laboratory. One method that we have used for manipulating global approach and global avoidance states is through the use of a raffle ticket procedure (Grimm, Markman, Maddox, & Baldwin, 2007; Maddox, Baldwin, & Markman, 2006a; Worthy, Brez, Markman, & Maddox, 2010). In the global approach condition, participants are informed that they will earn a raffle ticket into a drawing to win $50 if their performance exceeds a criterion. In the global avoidance condition, participants are given a raffle ticket for a drawing to win $50 upon entering the laboratory, but are informed that they will lose the ticket if their performance does not exceed a criterion. The bonus criterion and odds of winning the drawing are the same in both conditions. Thus, from an economic standpoint the situation is identical in both conditions; however the framing of the global motivation incentive is manipulated to create approach or avoidance scenarios. Local motivators, on the other hand, comprise the immediate (trial-by-trial) information that helps individuals maximize gains or minimize losses. To manipulate local motivational states, tasks are framed as gain maximization tasks (local approach) or loss minimization tasks (local avoidance). In the local approach condition, participants gain points on every trial in the task and attempt to maximize gains. In the local avoidance condition, participants lose points on every trial in the task and attempt to minimize losses. Critically, points gained and points lost are equated in such a way that the same overall level of performance is associated with the global motivational performance criterion needed to earn or retain the raffle ticket. Thus, at the level of the task a participant in any of the four possible experimental conditions is in an identical situation economically (earn raffle ticket by maximizing gains, earn raffle ticket by minimizing losses, avoid losing raffle ticket by maximizing gains, avoid losing raffle ticket by minimizing losses). Figure 1 presents a schematic representation of the global and local motivational framework that we propose. The two rows denote the global approach and avoidance motivational states and the two columns denote the local gains and losses motivational states. Our lab and others have argued that the influence of global and local motivation on task performance is interactive (Avnet & Higgins, 2003; Grimm, Markman, & Maddox, 2012; Grimm, et al., 2007; Higgins, 2000; Higgins, Chen Idson, Freitas, Spiegel, & Molden, 2003; Lee & Aaker, 2004; Maddox & Markman, 2010; Maddox, et al., 2006b; Markman, Baldwin, & Maddox, 2005; Shah, Higgins, & Friedman, 1998). We argue that a motivational match serves to up regulate effortful goal-directed processing whereas a motivational mismatch serves to down regulate effortful goal-directed processing, which, given the interactive nature of the systems, serves to enhance automatic habitual processing. Thus, we believe that the locus of these effects is broadly defined as prefrontal (Maddox, et al., 2006b). We hypothesize and find support for the prediction that a match between the global motivation and the local motivation leads to enhanced effortful task-directed cognitive control processing (see Figure 1A) whereas a mismatch leads to reduced effortful task-directed cognitive processing (and thus enhanced task-directed habitual processing; Figure 1B). It is important to note that in most cognitive research there are uncontrolled or poorly controlled global and local motivational states. At best a mild global approach motivational state is engaged by telling participants to “do their best” or by offering a small monetary bonus for good performance and a mild local “gains” motivational state is engaged by telling participants to maximize accuracy or maximize points (Maddox & Bohil, 1998). Motivation-Learning and Aging 5 Dissociable-Learning Systems and Task-Directed Motivation The theory that humans have multiple memory systems became widely accepted within the field of cognitive neuroscience during the 1980’s and 1990’s (Eichenbaum, 1997a, 1997b; Schacter, 1987; Squire, 1992; Squire, Knowlton, & Musen, 1993; Tulving, 2002). Since learning is a process of laying down memory traces, it is reasonable to argue that multiple learning systems exist that are capable of utilizing different types of memory traces associated with solving various tasks. Although dissociable-learning systems approaches have been explored in a number of domains including reasoning (Sloman, 1996), motor learning (Willingham, Nissen, & Bullemer, 1989), discrimination learning (Kendler & Kendler, 1970), and function learning (Hayes & Broadbent, 1988), the focus of the present chapter is on decision-making. Critically, we have demonstrated that one cannot develop a complete understanding of motivation and learning without acknowledging the existence of multiple learning systems and exploring system comparisons. Thus, in this chapter we examine goal-directed, cognitive control processes, which are the theme of this edited volume, in direct comparison with habitual, procedural processes to develop a complete view of the motivation-learning interface. Decision-Making Recently, there has been a surge of interest in examining the distinction between modelbased versus model-free decision-making systems and strategies (Blanco, et al., 2013; Daw, et al., 2011; Glascher, et al., 2010; Worthy, et al., 2012). Motivation plays a prominent role in distinguishing these two approaches to decision-making situations. Model-based decisionmaking is goal-directed, relies heavily on cognitive control and higher-level processing, and involves developing and utilizing a model of the environment that considers how each action can affect both immediate and future outcomes. Model-based decision-making is state-based because individuals are primarily motivated to perform actions that improve their future state (Glascher, et al., 2010). Model-free decision-making does not rely on cognitive control but instead on habitual, procedural-based processing and the motivational focus is centered on performing actions that lead to immediate reward or punishment. Actions that lead to immediate reward are reinforced and actions that lead to either immediate punishment or no reward are not. Model-free decision-making is reward-based because individuals are primarily motivated to perform actions that are followed directly by reward (Glascher, et al., 2010). Model-based and model-free decision-making processes, though somewhat overlapping and interactive, are thought to critically depend on separate neural systems with the weight given to each system varying across individuals and under different circumstances (Eppinger, Walter, Heekeren, & Li, 2013; Worthy, Cooper, Byrne, Gorlick, & Maddox, 2014). Areas of the ventral striatum are thought to be critical when generating reward prediction errors that are representative of immediate model-free rewards (Hare, O'Doherty, Camerer, Schultz, & Rangel, 2008; O’Doherty, 2004). In addition to ventral striatal regions that focus on available rewards, the intraparietal sulcus and lateral regions of the PFC, particularly the dorsolateral PFC (DLPFC) are critical in developing global model-based reward representations that map out the holistic structure of the reward space (Daw, et al., 2011; Glascher, et al., 2010; Smittenaar, FitzGerald, Romei, Wright, & Dolan, 2013). Given the critical regions underlying model-based and model-free processing, it should come as no surprise that recent studies have found an association between state-based and reward-based decision-making and working memory processes that are mediated by the DLPFC. Motivation-Learning and Aging 6 Here the presence of a dual task adversely affecting state-based decision making but not rewardbased decision-making (Blanco, et al., 2013; Daw, et al., 2011; Worthy, et al., 2012). Empirical Tests of the Motivation-Learning Interface in Decision-Making Enhanced cognitive control processing, or “trying harder”, is not always advantageous for efficient learning. When considering the interaction of global and local motivators on available cognitive resources, we predict a three-way interaction between global motivation, local motivation and learning system. Specifically, we predict that a motivational match (global and local approach or global and local avoidance) enhances task-directed cognitive control processes at the expense of task-directed procedural learning processes, and thus should enhance goal-directed learning, such as model-based decision-making, at the expense of procedural learning, such as model-free decision-making (Figure 1A). Analogously, we predict that a motivational mismatch (global approach and local loss minimization or global avoidance and local gain maximization) enhances task-directed procedural learning processes at the expense of task-directed cognitive control processes and thus should enhance procedural learning, such as model-free decision-making, at the expense of goal-directed learning, such as model-based decision-making (Figure 1B). We have found strong support for these predictions using a raffle ticket global motivation (seeking a ticket or saving a ticket) and local point motivation (gains vs. losses) in model-based and model-free decision-making and category learning (Maddox & Markman, 2010; Maddox, et al., 2006b; Markman, et al., 2005; Worthy, Maddox, & Markman, 2007). Other forms of global motivation have been examined (e.g, performance pressure and stereotype threat) as well as other goal-directed and procedural tasks (e.g., the Wisconsin Card Sorting Task, stimulus identification, and math problems) and the predictions from the motivation-learning framework were supported (Glass, Maddox, & Markman, 2011b; Maddox, Filoteo, Glass, & Markman, 2010a; Markman, Maddox, & Worthy, 2006; Worthy, Markman, & Maddox, 2009). Motivation-Learning Interface in Normal Aging One thing that is noticeably lacking in the normal aging literature is research focused on the influence of global and local motivational manipulations, their influence on task-directed motivational effects, and how this interacts with both goal-directed and habitually-mediated tasks. To our knowledge this three-factor motivational match framework (global motivation, local motivation, and task-directed motivation) has not been fully explored in normal aging, although one study examined the interactive effects of global and local motivators (Barber & Mather, 2013a) and a number of other studies have explored a global or a local motivational manipulation in isolation (Barber & Mather, 2013b; Braver, 2012; Braver & Barch, 2002; Braver et al., 2001; Castel et al., 2011; Ennis, Hess, & Smith, 2013; Frank & Kong, 2008; Freund, 2006; Hess, Auman, Colcombe, & Rahhal, 2003; Hess & Ennis, 2013; Hess, Leclerc, Swaim, & Weatherbee, 2009; Hess, Osowski, & Leclerc, 2005; Hess, Popham, Dennis, & Emery, 2013; Hess, Popham, Emery, & Elliott, 2013; Jimura & Braver, 2010; Jimura et al., 2011; McGillivray & Castel, 2011; Peters, Hess, Vastfjall, & Auman, 2007; Popham & Hess, 2013; SamanezLarkin et al., 2007; Westbrook, Kester, & Braver, 2013; Westbrook, Martins, Yarkoni, & Braver, 2012). However, few studies have explored task-directed motivation using behavioral tasks, such as decision-making tasks, that are identical in all respects except the nature of the optimal learning system and for which computational modeling approaches can be applied that provide Motivation-Learning and Aging 7 direct insights onto the locus of motivational effects (however see Maddox, Filoteo, & Huntington, 1998; Maddox, Pacheco, Reeves, Zhu, & Schnyer, 2010b). To examine this important issue we first explore how the well documented structural brain changes associated with normal aging affect task-directed processing in these dissociable learning systems (goal-directed and habitual) and associated tasks. We then briefly summarize the results from two recent studies conducted in our lab that examine the motivation-cognition interface in aging. Learning Systems and Task-Directed Motivation in Normal Aging A number of structural brain changes are well-documented in normal aging. For example, anatomical studies suggest that dramatic dopaminergic and volumetric declines across several brain regions are associated with normal aging, with the prefrontal cortices showing the largest volumetric declines in white and gray matter (Backman et al., 2000; Gunning-Dixon & Raz, 2003; Raz et al., 2005; Raz, Williamson, Gunning-Dixon, Head, & Acker, 2000). These structural and functional brain changes are associated with impairments in working memory and executive function both of which are critical for goal-direct learning, such as model-based decision-making (Bopp & Verhaeghen, 2005; Braver, 2012; Braver & Barch, 2002; Denburg, Tranel, & Bechara, 2005; Denburg et al., 2009; Filoteo & Maddox, 2004; Gunning-Dixon & Raz, 2003; Jimura, et al., 2011; MacPherson, Phillips, & Della Sala, 2002; Maddox, Chandrasekaran, Smayda, & Yi, 2013; Park et al., 2002; Racine, Barch, Braver, & Noelle, 2006; Samanez-Larkin, Kuhnen, Yoo, & Knutson, 2010; Schnyer et al., 2009; Titz & Verhaeghen, 2010; Wasylyshyn, Verhaeghen, & Sliwinski, 2011; Westbrook, et al., 2013; Westbrook, et al., 2012). For example, older adults show persistent robust deficits in tasks that critically rely on executive processes such as set-shifting during the Wisconsin Card Sort Task (Head, Kennedy, Rodrigue, & Raz, 2009). Structural and functional declines in the striatum are also well documented (Backman, et al., 2000; Gabrieli, 1995; Li, Lindenberger, & Sikstrom, 2001). These brain changes are likely associated with age-related deficits in procedural-based learning (McArdle, Ferrer-Caja, Hamagami, & Woodcock, 2002; Park, et al., 2002; Salthouse, 1991, 1994; Salthouse, Atkinson, & Berish, 2003). However in some domains, including model-free decision-making and category learning age-based procedural deficits are less robust, sometimes being present and others not (Filoteo & Maddox, 2004; Howard & Howard, 2001a; Howard & Howard, 1997, 2001b; Maddox, et al., 2013; Maddox, et al., 2010b; Raz, 2000; Raz et al., 2003; Samanez-Larkin, et al., 2007; Simon, Howard, & Howard, 2010; Worthy, Gorlick, Pacheco, Schnyer, & Maddox, 2011; Worthy, Otto, Doll, Byrne, & Maddox, in press). For example, older adults show intact early learning relative to younger adults during an implicit task where explicit processing of associations doesn’t contribute to performance (Howard et al., 2004). Model-based and model-free strategies are highly interactive and the robust cognitive declines associated with effortful controlled (model-based) processing along with less severe declines associated with automatic habitual (model-free) processing likely bias older adults toward the automatic habitual system. Importantly, the proposed bias toward automatic habitual processing, presented schematically in Figure 1C, should lead to age-related deficits in goaldirected tasks, such as model-based decision-making, but should lead to smaller deficits or possibly age-related advantages in habitual, procedural-mediated tasks, such as model-free decision-making. When applied to decision-making, we formalize this framework in a computational model that includes a weighting parameter that quantifies the bias toward the Motivation-Learning and Aging 8 model-based system. An additional advantage of this computational modeling approach is that multiple strategies can be formalized mathematically, applied to the behavioral data, and compared. It is important to be clear that we are not arguing that this framework broadly characterizes older adult cognition. Clearly the issue is much more complex. For example, in several domains normal aging does not lead to deficits in performance and in some cases actually leads to enhanced performance that may or may not be due to a shift in bias away from modelbased processing (for excellent examples of this perspective see Hess, 2014; Peters, et al., 2007). These include some aspects of value-driven episodic memory (Castel, et al., 2011; McGillivray & Castel, 2011), familiarity-based memory (Light, Patterson, Chung, & Healy, 2004), cognition in socio-emotional context (Blanchard-Fields, 2009; Blanchard-Fields, Jahnke, & Camp, 1995), as well as some aspects of category learning and decision making tasks (Glass, Chotibut, Pacheco, Schnyer, & Maddox, 2011a; Worthy, et al., 2011). Even so, this is a useful framework that seems to be applicable in many broad domains such as decision-making. Application 1: Task-Directed Motivation and Decision-Making in Normal Aging The effects of normal aging on decision-making are mixed. Some studies find deficits (Denburg, et al., 2005; Eppinger, et al., 2013; Kuhnen & Knutson, 2005; Mell et al., 2005; Mell et al., 2009; Samanez-Larkin et al., 2011), whereas others find advantages (Blanchard-Fields, 2009; Blanchard-Fields, et al., 1995; Cooper, Worthy, Gorlick, & Maddox, 2013; Grossmann et al., 2010; Worthy, et al., 2011; Worthy & Maddox, 2012). One way to address this apparent discrepancy in the literature is to determine the processing locus associated with optimal performance in each task and to examine whether older adults show deficits in some types of tasks but not others. The ideal approach is to use tasks that are identical in local and global motivation as well as surface features but for which the processing system that supports optimal decision-making is manipulated. We focus on state-based and reward-based decision making strategies that are optimally supported by effortful cognitive control and automatic habitual processing systems, respectively, and test the hypothesis that normal aging is associated with a shift in balance away from model-based processing toward model-free processing (see Figure 1C). This empirical approach should be complemented with the application of computational models. In a recent study from our lab (Worthy, et al., 2014) we examined the degree to which older and younger adults utilize model-free versus model-based reinforcement learning strategies using two tasks that are identical in all respects except in one task model-based processing was optimal (state-based decision-making) and in the other model-free processing was optimal (reward-based decision-making). In both tasks a mild global approach motivation was instantiated by informing participants that their goal was to maximize points gained and to exceed a performance goal (identical in all conditions), and a local approach motivation was instantiated by including only points gained for each option selection. Thus, as we were holding local and global motivation constant in this study we were most interested in investigating the inherent task-directed motivational states of older and younger adults. The reward structure associated with the state-based task is shown in Figure 2A. The decreasing option consistently provided larger rewards on each trial, but selecting the increasing option led to improvements in the participant’s state on future trials (i.e. the spot along the xaxis), while selecting the decreasing option led to declines in the participant’s state on future trials. The optimal strategy was to consistently select the increasing option which allowed Motivation-Learning and Aging 9 participants to reach the highest state, despite always providing smaller immediate rewards on each trial compared to the decreasing option. The reward structure associated with the reward-based task is shown in Figure 2B. Here, the optimal strategy was to consistently select the decreasing option even though selecting the increasing option led to improvements in state. The maximum value that could be obtained from repeatedly selecting the increasing option and reaching the highest state (55 units of oxygen) is smaller than the minimum value that could be obtained from simply selecting the decreasing option task on each trial (65 units of oxygen). Participants performed a four-choice variant in which two increasing and two decreasing options were included. On each trial, participants selected one of the four options and received the oxygen that was extracted which was added to a tank labeled “Cumulative”. A model-based strategy should lead to better performance in the state-based task compared to a model-free strategy because participants should be more likely to select the increasing option which improves their state on future trials. A model-free strategy should lead to better performance in the reward-based tasks compared to a model-based strategy because participants should be more likely to select the decreasing option which improves their current (and by extension, future) state. If older adults are more likely to utilize a model-free strategy compared to younger adults, then they should perform better on the reward-based tasks, but worse on the state-based tasks. We tested this hypothesis behaviorally by examining the total points earned in the task as well as by applying a recently developed HYBRID reinforcement learning model (HYBRID RL) (Worthy, et al., 2014). The HYBRID RL model provides unique insights into model-based and model-free strategies during learning as both of these systems are assessed together and the weight placed on the model-based system (w) is estimated (details can be found in Worthy, et al., 2014). Figure 3A displays the point total data and Figure 3B displays the w parameter estimates. As predicted, we found an age-related performance deficit in the four-option state based task, but an age-related performance advantage in the four-option reward-based task. Also as predicted, we found that younger adults placed greater weight on the output from the model-based system than older adults. We also examined the correlation between estimated w parameter values and the proportion of trials participants selected the Increasing option over the course of the task. There was a strong positive association in both the state-based (r=.63, p<.001) where these selections are advantageous and the reward-based task (r=.55, p<.001) where these selections are disadvantageous suggesting that model-based processing drives the selection of the increasing option regardless of its utility during the task. This study demonstrates the usefulness of rigorously defined tasks and computational models as tools for exploring age-based strategic changes that underlie performance. Application 2: Global Avoidance Motivation (Pressure) and State-Based Decision-Making in Normal Aging Application 2 reviews a study by Cooper, Worthy, Gorlick & Maddox (2013) that examined the effects of age and social pressure on performance in a two-option state-based decision making task (see Figure 2A). Here participants are told to attempt to exceed a performance criterion for a monetary bonus. However, their eligibility is a team effort that depends on their own success as well as that of a fictitious partner. Should one of them fail, neither will receive the bonus. Immediately prior to the start of the task, the participant is informed that their partner has succeeded and the fate of both of their monetary bonuses rests with the participant. Thus, performance pressure acts as a global avoidance motivator where the Motivation-Learning and Aging 10 participant’s goal is to avoid disappointing their partner. Combined with the local motivation to maximize points gained, global and local motivations are mismatched. Thus, we predict that pressure will lead to a performance decrement due to increased reliance on model-free processing. We tested this hypothesis behaviorally by examining the total points earned in the task as well as by applying the HYBRID RL model. Figure 3C displays the point total data and Figure 3D displays the w parameter estimate representing processing system bias. As predicted, we found a performance deficit in the pressure condition relative to the no-pressure condition in older adults that was due to a reduced reliance on the model-based system in the pressure condition. Somewhat surprisingly, we found a performance advantage in the pressure condition relative to the no-pressure condition in younger adults that was due to an increased reliance on the model-based system in the pressure condition. It is possible that younger adults viewed the pressure manipulation as a challenge in decision-making, and thus as a global approach motivation, whereas older adults viewed the pressure manipulation as a threat, and thus as a global avoidance motivation. General Discussion The common belief that motivation involves simply “trying harder” is at best simplistic and at worst is inaccurate. In this chapter we offer a three-factor framework for understanding the effects of motivation on cognitive processing and performance. We argue that global motivation and local motivation interact and drive the balance of processing between effortful, frontally-mediated cognitive control processes and automatic, striatally-mediated habitual procedural processes (Maddox & Markman, 2010). As outlined in Figures 1A and 1B, we propose that a regulatory match between the global and local motivational states affects the existing balance between cognitive control and habitual procedural processing. A regulatory match (global approach with local gains or global avoidance with local losses) shifts the bias toward cognitive control processing, whereas a regulatory mismatch (global approach with local losses or global avoidance with local gains) shifts the bias toward habitual procedural processing. Critically, the effect of this biasing on task performance depends upon the optimal strategy for solving the task. When the task is goal-directed, relying heavily on cognitive control processes, a regulatory match enhances performance whereas a regulatory mismatch impairs performance. On the other hand, when the task is reward-based, relying heavily on habitual procedural processes, a regulatory match impairs performance whereas a regulatory mismatch enhances performance (Figures 1A and 1B). In this chapter we began to apply this three-factor framework to normal aging. We conclude that to date the three-factor framework has not been fully explored in normal aging, but the handful of research has been conducted that explores aspects of the framework. Global motivational effects have been explored in normal aging in the realm of global avoidance during stereotype threat. The results are in general agreement with the regulatory match framework outlined in Figure 1. Specifically, older adults show poor goal-directed performance under global avoidance (stereotype threat) and local gains conditions, as would be expected from a regulatory mismatch (Barber & Mather, 2013b; Hess, et al., 2003; Popham & Hess, 2013). Local motivational effects have also been explored in normal aging and again, though not well controlled, the results are in general agreement with the regulatory match framework outlined in Figure 1. During habitual, procedural-based learning tasks with a mild global approach motivation to maximize performance, older adults are better at avoiding negative outcomes (regulatory mismatch) than approaching positive ones (regulatory match) (Frank & Motivation-Learning and Aging 11 Kong, 2008; Lighthall, Gorlick, Schoeke, Frank, & Mather, 2013; Marschner et al., 2005; Mell, et al., 2005; Pietschmann, Endrass, Czerwon, & Kathmann, 2011; Simon, et al., 2010). This follows from the three-factor framework as procedural based learning should be enhanced under a motivational mismatch (i.e., the avoid-negative outcome condition), and attenuated under a motivational match (i.e., the approach-positive outcome condition). Clearly though these effects need to be tested more rigorously in a controlled setting. To date only one study has explored the broad global/local motivation match framework in normal aging by manipulating both levels of motivation and even in this study the focus was on a goal-directed task, with no examination of habitual procedural-mediated processing (Barber & Mather, 2013a). Even so, this is the first study of its kind and provided strong support for the regulatory match hypothesis in normal aging. Specifically, Barber and Mather showed that stereotype threat (global avoidance) led to better performance for losses relative to gains whereas no threat led to better performance for gains relative to losses. Although progress has been made toward understanding the motivation-learning interface in normal aging, it is clear from this review that the literature is lacking in at least two ways. First, more emphasis should be placed on understanding task-directed biases across the lifespan. We propose that normal aging is associated with a shift in balance away from goal-directed toward reward-based processing (see Figure 1C). Unfortunately, the majority of extent research focuses on goal-directed tasks at the expense of an examination of reward-based processing, and the literature is almost completely devoid of research that explores goal-directed and rewardbased processing within the same experiment using tasks that are identical in all respects except for the cognitive processing system that mediates optimal processing (however see, Maddox, et al., 2010b; Worthy, et al., 2014; Worthy, et al., 2011; Worthy & Maddox, 2012). Second, the use of computational modeling techniques should be increased with the aim of better understanding the strategies being utilized to solve specific tasks. Despite the clear age-related structural and functional declines in brain and cognitive functioning with healthy aging, under some conditions older adults are remarkably adept at selecting cognitive strategies to optimize performance using the limited resources available. Although cognitive deficits are well established, in some cases older adults perform as well or better than younger adults (Glass, et al., 2011a; Maddox, et al., 2010b; Worthy, et al., 2014; Worthy, et al., 2011; Worthy & Maddox, 2012). Of course, under many other conditions older adults appear unable to utilize the optimal strategy for solving a task and instead fall back on a simpler, sub-optimal strategy (Filoteo & Maddox, 2004; Maddox, et al., 2013; Maddox, et al., 1998). Without including computational models that provide insights onto these age-based changes in strategy selection, these findings are often deemed anomalous or